Tech Arsenal 1/Tech Arsenal (Arsenal Computer).ISO/tek-02/extend5.zip/EXTEND.DOC

Text File | 1990-11-19 | 14KB | 245 lines

(*
                                  EXTEND.PAS
                                  Release 5
                              November 19, 1990
                              by Scott Bussinger
                             Compuserve 72247,2671


  MS-DOS and PC-DOS Turbo Pascal 3.0 and higher only allow up to 15 files to
be open at the same time, due to limitations in DOS.  This file shows you how
to have up to 96 files open simultaneously under DOS 2.0 or 2.1, or 252 files
open simultaneously under DOS 3.0 or higher.  Below is a description of how to
use this technique, followed by a technical explanation of the implementation,
for those who are interested.

  The implementation of this technique for DOS 2.X is based on a technique
originally written by Randy Forgaard.  The interrupt handler and DOS 3.X
implementation was written by Scott Bussinger with input and help from several
others including Randy Forgaard, Bela Lubkin, Kim Kokkonen and Brian Foley.

  The early releases of this software used a traditional version numbering
system (e.g. 1.1, 1.2, 2.1, etc.) and then I starting using TLIB (a source code
control program) to manage the software and changed the numbering scheme to be
compatible with that program.  The versions are now numbered sequentially with
the first version (release 1) corresponding to the old version 3.2.  The
documentation got so confusing that I've liberally re-editted the documentation
files to eliminate the old references.  If anyone cares about the old comments
and release history, grab an old copy of the source code and look there (it's
not very interesting to anyone but me anyway).

TO USE THIS TECHNIQUE:

  You simply include the EXTEND unit in the USES statement in the source code
of your main program.  You will have to either compile the EXTEND unit
separately or use Turbo Pascal's project compiling features to Make or Build
your program (which will compile the EXTEND unit automatically).  The
EXTEND unit should be the first unit in your USES statement, following only
the DOS unit (which is USEd by the EXTEND unit).  The unit automatically
installs itself and will uninstall itself when your program ends or an error
is detected.  See the demo program below for an example of using the EXTEND
unit.

  The EXTEND.PAS file will need to be available available to be compiled if
the Make or Build features are used to compile the program.  The EXTEND.ASM
is the source code for the assembly language interrupt handler used when
running under DOS 2.  The EXTEND.OBJ file is the assembled object code for
EXTEND.ASM and was compiled using TASM.

  You must edit your CONFIG.SYS file (see the DOS manual for details), so
that it includes a line that says "FILES=XXX".  XXX should be a number that is
at least 3 greater than the maximum number of files that will be in use
(opened by RESET, REWRITE or APPEND) at the same time during the execution of
your program (larger values will provide no additional benefit, with respect
to your individual program), and should not exceed 99 (for DOS 2.0/2.1) or 255
(for DOS 3.0 and higher).  Under any version of DOS, the minimum allowable
value for XXX is 8.  Then, reboot your computer so that the FILES=XXX
parameter takes hold.  This same change to the CONFIG.SYS file will have to be
made on any machine in which your program is to run.

  Running the sample program at the bottom of this file will tell you the
maximum number of files you can have open at once (usually 3 less than the
number of files specified in the CONFIG.SYS file as discussed above, unless a
resident program has opened some files and hasn't closed them yet).

THE TECHNICAL DETAILS:

  Much of the following information is not documented in the DOS Technical
Reference manual.

  Under DOS 1.0 and 1.1, all files were accessed via File Control Blocks
(FCB's).  There was no limit to the number of FCB's that a program could use,
so there was no limit to the number of files open simultaneously.

  Under DOS 2.0 and higher, an alternate (and preferable) method of accessing
files was introduced, using a 2-byte integer called a "handle" to refer to a
file.  A "handle" file is described using a data structure called a Device
Control Block (DCB).  However, DOS provides the storage space for all DCB's,
rather than having the application program store the DCB's, so the number of
available DCB's is limited to the amount of space that DOS has set aside for
them.  The maximum number of files/devices that can be open simultaneously, on
the whole computer, is the number of slots available in the DCB Table created
by DOS.  The DCB's in the DCB Table are consecutively numbered, starting with
0.  DCB's 0, 1, and 2 are predefined by DOS to correspond to the AUX, CON, and
PRN devices, respectively.  All remaining DCB's in the DCB Table are available
for files or devices used by application programs.

  So that I/O redirection can be supported, the DCB numbers are not used
directly when accessing files.  Instead, a file "handle" is used.  A "handle"
is an index into a 20-byte array, called the Handle Table, located at offset
18H of the Program Segment Prefix (PSP) for a program (for a general
discussion of the PSP, see the DOS Technical Reference manual).  Each element
of the Handle Table is the DCB number of a file or device.  The value at index
"handle" in the Handle Table is the DCB number of the file or device that
implements that file handle.  Thus, if the value 8 is in the 6th byte of the
Handle Table, the handle "6" refers to the file (or device) described by the
DCB in slot 8 of the DCB Table.  If a handle is not currently being used, its
entry in the Handle Table is FFH.  DOS predefines the first 5 handles to be
primary input, primary output, error, auxiliary, and printer, so the first 5
entries in the Handle Table are 1, 1, 1, 0, and 2, corresponding to the DCB
numbers for the CON (1), AUX (0), and PRN (2) devices.  This leaves only 15
available handles for opening files (or new devices).

  Every time a new handle file is opened, a new handle gets used.  Since there
are only 20 slots available in the Handle Table for a program, DOS only allows
a "process" to have a maximum of 20 file handles in use simultaneously (and the
first 5 entries are predefined, as just noted, unless those handles get closed
and reused).  Every new handle file requires a unique handle, so only 20
files/devices can be open at the same time by a single process (unless FCB's
are used).  (A "process" is any program spawned using the DOS EXEC function
call.  A process can be invoked by COMMAND.COM, or by another program.)  There
can be many more than 20 DCB's in the DCB Table, so the real limitation is in
the size of the Handle Table in the PSP.

  The size of the DCB Table (i.e., the maximum number of files/devices that
can be open simultaneously in the whole computer) is controlled by the
FILES=XXX entry in the CONFIG.SYS file.  The minimum number of slots is 8.
Under DOS 2.0/2.1, the maximum number is 99, and under DOS 3.0 and higher,
the maximum is 255.  As previously mentioned, the first three of these DCB
slots are occupied by the AUX, CON, and PRN devices.

  A single program can use all of the DCB's in the DCB Table (except for the 3
reserved by DOS) all on its own, by effectively bypassing the Handle Table in
the PSP, except on a temporary basis.  The program can accomplish this feat by
using, say, only one entry in the Handle Table for all of its files.  Instead
of allowing DOS to store the DCB numbers in the Handle Table, the program can
store these numbers elsewhere.  Then, to manipulate a file using DOS, the
program can temporarily put the DCB number of that file into a designated slot
in the Handle Table, pass the index of that table slot (i.e., that "handle")
to DOS, and DOS will operate on that handle/DCB number.  After the DOS call,
the program can remove that DCB number from the designated Handle Table slot,
freeing up that handle for use in another DOS call for another file.  In this
way, DOS can be fooled into accessing up to 96 (or 252) different files/devices
using a single handle entry in the Handle Table.

  To obtain the address of the Handle Table, which is at offset 18H in the
PSP, the program needs to find the address of its PSP.  Under Turbo Pascal
4.0 and later, the segment address of the PSP is stored in a pre-defined
variable called PrefixSeg setup by Turbo's standard initialization code. All
of the manipulation of the DCB's and handles is dealt with by ExtendHandler
which is an interrupt handler which intercepts all DOS functions calls (INT
21) and checks to see if the function involves a handle number. If the
function returns a handle (e.g. an Open File) it then returns the DCB to the
calling program in place of a handle. It adds an constant offset of 5 to the
DCB so that ExHandler can distinguish between DCB's and calls refering to the
5 automatically assigned standard handles. If the function invoked passes a
handle to DOS (e.g. Read File) it checks to see if it refers to a standard
handle and if so it passes control normally. If not, it subtracts the constant
offset of 5 and temporarily usurps the use of handle 19. It places the DCB
into the the slot in the handle table for handle 19 and changes the handle in
the registers to 19. The previous DCB in handle 19 is saved and restored
afterwards (this probably isn't necessary, but made me feel better).

  Note that these routines support all DOS functions up through DOS 3.3
including those not normally used by Turbo.  The exception is function
$46 (FORCEDUP). This call is rarely used, but incompatible with the methods
used by ExHandler.  The only use that I know of for this function is to
redirect the standard handles and since the standard handles are effectively
skipped by these routines, there is no problem in using handles 0-4 as the
second handle in a FORCEDUP call.  No other handles are allowed and will
automatically result in an error being returned.  In other words, go ahead and
redirect the standard handles, but don't try it with anything else.

  All of this hocus-pocus is required under DOS 2.X, but DOS 3.X is more
intelligent and actually has some support for moving the handle table from the
PSP to elsewhere in memory. So when running under DOS 3.0 to 3.2, the EXTEND
unit uses a slightly documented capability to install a new, larger handle
table. Essentially, Microsoft started to permit a standard way of creating a
new handle table. This information is has been seen in at least one Microsoft
document and has worked on every DOS I've seen or heard about. There are now
two fields in the PSP which contain a pointer to the beginning of the handle
table and a byte specifying the number of available handles. This method is
much less kludgey and solves some problems with running programs under
debuggers. The space for the new handle table is allocated out of free DOS
memory (not in the memory allocated to the program by DOS when it started up
the program). If no free memory is available, then the current program is
shrunk down enough to allow the memory to be allocated. Using this memory
solves problems with swapping EXEC programs and TSR's. When the program is
finished, that memory is automatically released by DOS.

  As of DOS version 3.3, Microsoft actually added a real way to extend the
number of handles available to a program and EXTEND.PAS takes advantage of
that method when possible to ensure maximum compatibility in every
environment.

  There is one incompatibility that I know of with EXTEND.  That is
sub-functions 4 and 5 of DOS function $44 (IOCTL) when running under DOS 2.
These calls send and receive control strings from block devices.  These two
sub-functions use registers differently than all other sub-functions of IOCTL.
I can't imagine a use for these two functions in a Pascal program, so I
haven't bothered to modify the interrupt handler to identify this case.  If it
is a problem, let me know and I may fix it.

*)

{  Example program  --  This program opens as many Text files as it can, until
 DOS runs out of room in its DCB Table.  It then reports how many files were
 successfully opened, writes a line to each of them, then closes and erases
 each of them.  Note: The value of the FILES=XXX parameter in the CONFIG.SYS
 file must be set to an appropriate value (see above).  If you change the
 FILES=XXX value, be sure to reboot before running this program.

 This program takes a while to run, due to the heavy disk I/O, so running it on
 a hard disk (or, even better, a RAM disk) is recommended.  Make sure that you
 are running the program in a subdirectory, so that you don't run up against
 the DOS limit on the number of allowable files in the root directory of a
 drive. }

program Test;

uses Extend;       { <--- This is the magic line that does everything }

const MaxCount = 255;

type FileArray = array[1..MaxCount] of text;

var Count: integer;
    F: ^FileArray;
    I: integer;
    Num: string[6];

begin
new(F);            { Use heap because of large size of this array }
writeln('Opening files...');
I := 0;
repeat
  inc(I);
  str(I,Num);
  assign(F^[I],'junk' + num + '.txt');
  {$I-}
  rewrite(F^[I])
  {$I+}
until ioresult <> 0;
Count := I - 1;
writeln('Successfully opened ', Count, ' files at the same time.  Writing to each file...');
for I := 1 to Count do
  writeln(F^[I], 'This is a test.');
writeln('Closing and erasing each file...');
for I := 1 to Count do
  begin
  close(F^[I]);
  erase(F^[I])
  end;
writeln('Done.')
end.